The origin of the modern keyboard as the primary method for inputting text from a human to a machine dates back to early typewriters in the 19th century. As computers were developed, it was a natural evolution to adapt the typewriter keyboard to use as the primary method for inputting text. For a skilled typist, it has remained the fastest way possible to input text into the computer.
With ongoing efforts to make computers smaller and more portable, the physical keyboard has become one of the most significant limiting factors in just how small a device can become: the physical size of the human finger is not something computer designers could change. As a result, computers for certain portable applications have been designed without a physical keyboard, and use touch-screen based input methods as the primary form of human computer interface. (This is also the case for some applications where people are physically unable to use a keyboard, such as persons with physical disabilities.)
There are two primary requirements for touch-screen input methods which frequently conflict with each other. The method of input must be as fast as possible and at the same time the method of input must take as little of the display screen as possible. Unfortunately, as the space taken up for input on the display screen is decreased, it becomes difficult to increase speed without adversely affecting accuracy.
In spite of a recent surge in the market for pen-based computing devices, most people who must generate text still do so with a standard keyboard. In fact, an entire industry has sprung up that provides portable keyboards for pen-based computers that were designed to be keyboard-less! To date, pen-based computing has not replaced conventional portable laptop computers as was originally forecast, for the simple reason that text input on pen-based computers is too slow. Even the recently introduced “Tablet PC,” largely due to the lack of a sufficiently accurate handwriting recognition engine, serves largely as a way to store and retrieve “digital ink”—actual graphic images of handwriting traced on the touch-screen by the user—as opposed to recognizing what was handwritten and converting to computerized text.
Analogous to one-finger typing, the current state-of-the art for inputting using a virtual keyboard is called “point and tap”. A stylus is moved from letter to letter and pressed down on the desired key to select it. This results in the need to always lift and set down the stylus, slowing input. Cursive handwriting was invented to allow a better (and faster) flow from letter to letter and reduce the number of pen (or quill) lifts. In a similar way, the current invention reduces the number of taps required when inputting using an on-screen keyboard, thus speeding text entry.
The natural way for a human to create text on anything other than a machine is to “write” it by hand. Accordingly, with the advent of touch-screen computers, it is not surprising that handwriting recognition software was developed to allow a user to enter text by writing on the screen of the computer. However natural, handwriting is slow. Each letter requires several strokes of the stylus, making it very inefficient. Further, with varying handwriting styles, accuracy of this software is still below user-acceptance levels (see MacKenzie, I. S., & Chang, L. (1999), A performance comparison of two handwriting recognizers. Interacting with Computers, 11, 283–297.). As mentioned above, the reliance on the use of “digital ink” by even the “latest and greatest” of computer touch-screen technology, the Tablet PC, provides clear evidence that handwriting recognition is still not good enough to satisfy most users. Furthermore, even if a completely accurate method of handwriting recognition were available, handwriting itself is simply too slow and tiring (especially on a touch-screen) to provide a satisfactory input method.
Some methods make the job easier for the software by requiring the user to handwrite letters in a simplified way (see Goldberg, US Patent Application 20020009227, Unistrokes; or as popularized by Palm Computing in their commercial product titled “Graffiti”). Advantages of this method are that each character is sufficiently unique as to be easily recognized by the software, that each character is drawn as a single stroke, and no virtual keyboard is required on the screen. Disadvantages of this method are it requires the user to learn a new writing method and still requires the stylus to be set-down and lifted once for each individual character (thus slowing text entry).
Again, it was a natural evolution for the idea of a keyboard to be carried on into the virtual world of the computer display. Auer et al., in U.S. Pat. No. 4,725,694, describe a system wherein one or more images of simulated keyboards are displayed on a touch-sensitive screen of a computer, and in response to the touching of the simulated keys, generate appropriate control signals. In a later refinement of this concept, the image of the keyboard is displayed floating above other applications running on the computer, rather than occupying a dedicated portion of the screen. The user interacts with this “on-screen keyboard” or “virtual keyboard” by either directing a cursor pointer over it, or directly touching the keys via a touch screen using a finger or stylus. On-screen keyboards have been primarily used for devices which lack a standard keyboard, such as certain public information kiosks and personal digital assistants (PDAs), handheld computers that are too small to accommodate a physical keyboard. They are also frequently used by individuals with disabilities that prevent them from using a physical keyboard.
There are two large shortcomings of on-screen keyboards: first they take up valuable screen space on the computer needed for whatever task is requiring text input. Second, and more importantly, they are slow because the user is forced to tap one letter at a time—effectively reducing the user to input text in a way that is analogous to single finger typing on a regular physical keyboard.
In an effort to address the slow rate of typing with on-screen keyboards, predictive software was developed which, based on preceding words and on the initial letters typed for the current word, attempts to predict what word is being typed and presents the user with a list of word or phrase choices that they can select as a quicker alternative to completing the word or phrase letter by letter. Due to the need to divert attention from the task at hand (typing) in order to scan the prediction list and determine whether the intended word has been offered as a choice, this “Word Prediction” scheme offers only a marginal increase in speed of text entry, if any (depending on the user).
As the size of the on-screen keyboard is reduced beyond a certain point, the speed of text entry is sharply reduced. This is due to the requirement for increased accuracy and dexterity in hitting the smaller targets. Various schemes have been developed to minimize the size of the keyboard, yet still maintain accuracy without unduly sacrificing speed of entry.
Grover et al., in U.S. Pat. No. 5,818,437, describe a system that reduces the number of distinct keys required by assigning multiple letters on each key. This allows, for a given size of keyboard, relatively larger individual keys which are therefore easier to hit accurately, thus allowing the user to type more quickly. Text entry in this system is word-based, so that disambiguation software uses a database of words to analyze each sequence of keystrokes and determine the most likely word (or words) corresponding to the sequence, and consequently determine which letter was actually intended by each ambiguous keystroke. While the system of Grover et al. makes it easier to hit an intended key by virtue of reducing the total number of keys and enlarging the individual keys, when implemented on a touch-screen device, it still requires the user to lift the stylus and set it down for each letter entered, significantly slowing down text entry. Furthermore, this approach requires the user to a very unfamiliar keyboard layout in which completely unrelated letters are grouped together on a single key. Even when letters are grouped according to an “alphabetic” layout (as on the keys of cellular telephone), compared to the standard “QWERTY” keyboard, the arrangement is unfamiliar for the majority of individuals and further slows the text entry process.
Lee, in U.S. Pat. No. 6,292,179, describes another system that reduces the number of distinct keys required on a touch screen keyboard by assigning multiple letters on each key, and determining which letter associated with a contacted key is intended by determining the direction in which the stylus is moved after contacting a key. Each letter associated with a key is further associated with a range of directions in which the point of contact can be moved. Lee's method also allows each key of a given keyboard to be relatively larger, and therefore easier to initially contact, since multiple letters are combined into a single key and a smaller total number of keys is therefore required. However, the user is still required to tap once for each desired letter, and is further required to move the point of contact in a particular direction before lifting the stylus and breaking contact with the screen.
Kaehler, in U.S. Pat. No. 5,128,672, describes another system designed to reduce the number of distinct keys that are required for a touch screen keyboard by displaying at any given time only a subset of the total set of characters that can be entered. The system attempts to determine the subset of characters comprising the most likely next characters to be entered, based on the previous character entered or the positioning of the text insertion point. When the desired character does not appear, the user must manually switch to a different keyboard to locate and enter the desired character. The large number of different (and constantly changing) partial keyboards would tend to make this a slow and frustrating input method for the majority of users.
Vargas, in U.S. Pat. No. 5,748,512, attempted to reduce the need for accuracy on a touch screen keyboard (and therefore increase speed) by considering two adjacent keys as possible candidates when a key is not activated in its central area. Based on the actual location at which the keyboard was contacted relative to the three keys, combined with the statistical analysis of preceding characters in the word being entered (if any), and optionally also using information from a word prediction engine, the system determines the most likely of the three possible candidate characters and displays it as the character to be input in response to the activation. However, since each character, once input, forms the basis for the prediction of subsequent characters, when a character prediction is incorrect, it must be corrected by the user before the can proceed to type the next character. In the system as described, this is done by maintaining contact with the keyboard during an activation, observing whether the predicted character is correct, and if necessary, sliding the point of contact in the direction of the actual intended letter. The advantage of this invention, as described, is that it enables the user to use something blunt (such as a fingertip, rather than a stylus) to activate keyboard keys that are actually smaller than the instrument of activation. However, the nature of the feedback provided to the user and the consequent need to observe the result of each keystroke and correct it before moving on to the next keystroke, creates a system that generally would slow the rate of text input considerably.
Robinson et al., in international patent publication WO 00/74240 A1, describe a text input system for touch-screen devices that includes a keyboard with an auto-correcting region that includes the set of keys that are associated with letters. The advantage of the system is that, for words that are included in the system database, the user does not need to contact within the region of the key associated with desired letter, but instead need only tap in the neighborhood of the key. The user taps the keyboard once for each letter in the word being entered, and the system records the location of each contact. The sequence of contacts is then compared with the key locations associated with words in the database, and the most likely one or more matching words are presented to the user for selection. This system is a significant improvement over previous approaches in that it enables the user to type much more quickly on a small keyboard because it is no longer necessary to precisely contact within the region of each intended key. However, for each key activation, the user still needs to touch down on the screen with control, targeting the intended key, then lift the stylus from the screen and move to target the next key. The additional movements of lifting and setting down the stylus for each letter, combined with the additional effort required to control the relocation of the stylus when it is not in contact with the screen, result in significantly slowing down the input process compared to the system of the present invention.
Another factor in slowing text entry on touch-screen keyboards was the time it takes to lift the stylus from the screen and then bring it back down between each key selection (“tapping”). U.S. Pat. No. 5,574,482 (Niemeier) discloses a method for data input on a touch sensitive screen. The Niemeier patent teaches having what are described as computer generated “temporary” adjacent keys which can be made to appear on top of a standard keyboard layout. When the user touches a key, selecting a first letter, one or more temporary keys are displayed adjacent to the contacted key as long as the initial contact is maintained. A second letter (or group of letters) that is displayed on an adjacent temporary key can then be selected by making what is described as a “wiping” motion in which one's finger or a stylus is slid from the first selected key to the adjacent temporary key. This teaching enables two (or more) adjacent letters to be input without lifting the stylus from the screen, approximately cutting in half the number of times the stylus needs to be lifted from the touch screen. The “temporary” adjacent keys create artificial groupings of the most probable letters to provide more opportunity for “wiping” input.
However, the method described by Niemeier has several significant drawbacks. One is that the user needs to either memorize 26 new “temporary” sub-keyboards that appear when each of the 26 letter keys are contacted (creating a significant learning curve in order to use the system effectively), or else the user needs to pause to observe whether and where the next desired key may have appeared on a temporary key, which would likely negate any speed advantage that the “tap and slide” method would provide. The situation becomes worse when, in order to increase the likelihood that the desired letter will appear, a dictionary or other database is used to alter the temporary keys that are displayed for each key based on the context of the previous letters that that were selected prior to activating the key. Furthermore, as described above, the system is limited to sliding to at most one additional key beyond the initially selected key, unless the subsequent letter also happens to appear on a key adjacent to the selected temporary key. Also, the number of temporary keys that can be displayed (and therefore selected) is limited to the number that can be displayed around the circumference of a single key (six for standard key arrangements as proposed by Niemeier, up to a maximum of eight). Furthermore, since the temporary keys are only displayed while the stylus (or finger) is actually in contact with the screen, a significant number of the keys which might be displayed would likely be partially or totally obscured by the stylus and hand of the user. Also, the system proposed by Niemeier includes a significant amount of space in between active keys (used for capitalization and spacing), reducing the size of the actual key targets, each of which must be contacted in the interior of the defined key area in order to effect an activation.
Van Kleeck, in U.S. Pat. No. 6,008,799, describes a somewhat similar system, wherein the “temporary keys” are never actually displayed, but where each of the four vowels “o”, “e”, “i” and “a” are implicitly associated with each letter key, and can be appended by tapping a character button and dragging the pen in either the north, south, east or west direction. While this approach is easier to learn than Niemeier's (since only the four vowels and their associated directions must be learned), and it is not affected by potential visual interference by the user's hand (since no temporary keys are actually displayed), it is limited to only being able to add one of the four particular letters following a key selection, and thus does not offer a very significant improvement over a standard keyboard.
Perlin describes a method in U.S. Pat. No. 6,031,525 where the stylus is never lifted from the touch screen, but rather directed from a middle neutral point (a “resting zone” in Perlin's terminology) to one of a number of surrounding zones, each of which contains a plurality of letters. The user is required to contact the screen in the resting zone and then perform a continuous sliding motion which, for each letter to be entered, goes from the resting zone out into the adjacent zone that contains the desired character, and then indicates which character contained in the zone is intended by either returning directly to the resting zone or by first traveling to a different zone before returning to the resting zone. Thus, the desired character is indicated by the combination of the zone first entered from the resting zone, combined with the zone from which the resting zone is again re-entered. As a result, the user is required to make a minimum of 2 and more commonly 3 strokes of the stylus to indicate each desired letter. This method allows for a smaller keyboard, but requires multiple strokes per letter which dramatically reduces the speed of text entry.
Another common approach in speeding text entry is to modify the layout of characters on the keys of an on-screen keyboard (see www.fitaly.com). Common letters are placed near the center of the onscreen keyboard, and common letter pairs are made adjacent. In this way, stylus movement between letters is minimized, thus speeding text entry. The disadvantage of this method is it requires the user to learn a new keyboard layout, and again, the user is still required to lift the stylus between each key activation, essentially reducing the user to a one-finger typist.
Each of the above methods has the potential to speed text entry by some degree and/or decrease the amount of on-screen real estate required for input. However, text entry via existing on-screen keyboards and handwriting recognition techniques are still generally slower than using a physical keyboard. Handwriting itself is simply too slow, and the recognition accuracy is still not up to acceptable standards. All of the keyboard-based methods described above require one or more separate and distinct actions for the input of each individual letter. Most of the proposed keyboard systems are based on tapping some type of key, so that generating a single character requires that the stylus be appropriately positioned, the on-screen keyboard be contacted, and the stylus lifted from the screen again before proceeding to input the next character.
The fundamental problem is that, particularly for devices that must utilize touch-screen keyboards that are reduced in size, the additional actions of lifting the stylus and bringing it back into contact with the screen in a controlled manner considerably slow down the input process. Other methods which have been proposed to reduce the number of times the stylus must be lifted from the screen (such as Niemeier and Perlin) still fail to provide a method that can significantly speed text entry. Niemeier's proposed system adds too much complexity by requiring the user to react to a constantly changing keyboard layout, and limits the number of characters that can be selected following an initial character selection. Perlin's approach fails because it requires the user to execute too many distinct stylus movements to enter each letter.